Building a Multimodal LLM-Based Search Assistant Chatbot to Enhance Housing Search
Tetiana Torovets and Giulio Santo and Lucas Cardozo • Location: TUECHTIG • Back to Haystack EU 2024
“QuintoAndar Group is the largest housing platform in Latin America, leveraging cutting-edge AI technologies to streamline the housing search process, reducing paperwork and increasing accessibility. To elevate our users’ experience even further, we developed a groundbreaking search experience by adopting contrastive vision-language models and Large Language Models (LLMs) to build a multimodal chat-based search assistant. We adopted contrastive models as embedding generators to enable multimodal search capabilities, and utilized LLMs to interpret and extract user preferences within a conversational interface. In this presentation, we share our insights and experiences from this development process. We discuss the adopted architecture and how we reconcile good engineering practices while working with LLMs. We also discuss the integration of our traditional search mechanism with multimodal capabilities and explain how we use a chatbot to guide this enhanced search assistant.”
Tetiana Torovets
QuintoAndarTetiana Torovets is the Head of Data Science at QuintoAndar, where she leads initiatives in Search, Personalization, and Generative AI. With over a decade of experience in Data Science, Tetiana has a strong background in fintech and fashion industries. She joined QuintoAndar six months ago, bringing her expertise in creating data-driven products that enhance user experiences. Tetiana's journey in search began three years ago at thredUP, where she focused on developing search and recommendation systems.
Giulio Santo
QuintoAndarGiulio Cesare Mastrocinque Santo is a Staff Data Scientist at QuintoAndar, working on machine learning and AI projects, including predictive models for credit, search, ranking, and computer vision. Before QuintoAndar, Giulio worked on machine learning projects at Nubank and Petlove. He holds a B.S. in Electrical Engineering and an M.S. in Systems Engineering from the University of São Paulo.
Lucas Cardozo
QuintoAndarLucas is a Staff ML engineer at QuintoAndar with an interesting journey that started in Biology and later evolved into the world of data. At QuintoAndar, Lucas had the opportunity to experience both the ML Platform side by building tools, infrastructure and processes for QuintoAndar DS practitioners, as well as the Product side, actively working with house recommendations and generative AI for search in real estate.